Data distributed parallel applications like MapReduce are widely used. Focusing on the issues such as low execution efficiency and high cost of such applications, a case analysis of Hadoop was given. Firstly, based on the analyses of the execution processes of such applications, it was found that the data volume, the numbers of the nodes and tasks were the main factors that affected their execution efficiency. Secondly, the impacts of the factors mentioned above on the execution efficiency of an application were explored. Finally, based on a set of experiments, two important novel rules were derived as follows. Given a specific volume of data, the execution efficiency of a data distributed parallel application could not be improved remarkably only by increasing the number of nodes, but the execution cost would raise on the contrary. However, when the number of tasks was nearly equal to that of the nodes, a higher efficiency and lower cost could be got for such an application. The conclusions are useful for users to optimize their data distributed parallel applications and to estimate the necessary computing resources to be rented in a cloud environment.

Focusing on the higher ratio of processor utilization and lower execution cost of a scientific workflow in cloud, a policy of execution optimization based on task cluster aggregation was proposed. First, the tasks were reasonably replicated and aggregated into several clusters. Therefore, the key tasks could be scheduled as early as possible. Then, the task clusters were aggregated again to facilitate the spare time among the tasks in the task cluster. The experimental results show that the proposed policy can improve the parallelism of workflow tasks, advance the earliest finish time of the whole workflow and it has a significant effect in improving the utilization ratio of processors and lowering the cost of workflow execution.